# Small Language Models

MiniRAG is a retrieval-augmented generation system designed for small language models, aimed at simplifying RAG processes and enhancing efficiency. It addresses the performance limitations of small models within traditional RAG frameworks through a semantically aware heterogeneous graph indexing mechanism and lightweight topological enhanced retrieval methods. This model shows significant advantages in resource-constrained scenarios, such as on mobile devices or edge computing environments. Its open-source nature allows for easy adoption and improvement within the developer community.

Phi Open Models, offered by Microsoft Azure, redefine the potential of small language models (SLMs) with exceptional performance, affordability, and low latency. While maintaining a compact size, the Phi models provide robust AI capabilities, reducing resource consumption and ensuring cost-effective generative AI deployments. The development of Phi models adheres to Microsoft's AI principles, which include responsibility, transparency, fairness, reliability and safety, privacy, and inclusiveness.

SLM_Survey is a dedicated research project focused on small language models (SLMs), aiming to provide an in-depth understanding and technical assessment of these models through research and measurement. The project covers transformer-based, decoder-only language models with parameter sizes ranging from 100M to 5B. By investigating 59 state-of-the-art open-source SLMs, it analyzes their technological innovations and evaluates their capabilities across multiple domains, including common-sense reasoning, context learning, mathematics, and programming. Additionally, runtime costs such as inference latency and memory usage have been benchmarked. This research is of significant value in advancing the study of SLMs.